dir.to.work = "C:\\Users\\Yoav\\Documents\\gdrive.mail.tau.ac.il\\Other computers\\My Laptop\\"
source(paste0(dir.to.work, "work\\resources\\stasExamples\\R\\yba.funcs.R"))
# The data's folder (directory)
dir.to.data = "C:\\Users\\Yoav\\OneDrive - Tel-Aviv University\\Documents\\bigfiles\\"
dir.out <- paste0(dir.to.data, "ageformva\\processed")

Overview of the study

  • In previous research, participants who read behavioral evaluative information about a young man and an old man showed a strong pro-young bias in their automatic evaluation of those targets.

    • More generally, group membership (gender, age, race) had a larger influence on automatic evaluation of individual group members than on non-automatic evaluation.

    In this image, we see the preference for a positively-portrayed target over a negatively-portrayed target, in the self-repotr evaluation (on the left) and in the IAT (on the right). The x-axis is the behavioral information condition. For example, when x is 10, it means that the positively-portrayed target was characterized by 13 positive and 5 negative behaviors, whereas the negatively-portrayed was characterized by 13 negative and 5 positive behaviors. In the present experiment, we will describe the positively-portrayed target with 8 positive and 4 negative behaviors, and the opposite for the negatively-portrayed target. That’s a difference about 8 behaviors.

  • The purpose of the present experiment was to investigate possible causes for the group effect. We will try to estimate:

    • The direct effect of mere visual cues on evaluation.
    • The effect of group-membership knowledge after encoding
    • The effect of group-membership knowledge during encoding
    • The effect of visual cues during encoding.
  • Overview of the procedure:

    • The experiments consisted of four parts: behavioural information, name-photo matching, evaluation (both with the IAT and with self-report), memory test.
    • In the behavioral information task, the positively-portrayed target was characterized by 8 positive and 4 negative behaviors, and the negatively-portrayed target was characterized by 4 positive and 8 negative behaviors.
    • We manipulated whether photos appeared in Part 1 or in Part 3.
    • We manipulated whether the age of the targets was provided in Part 1 or not.
    • The presentation conditions in the experiment:
      • Baseline. Part 1: Only names; Part 3: Only names. (age groups are provided only in the name-photo matching task).

      • Age1. Part 1: Name + age; Part 3: Only names.

      • *Photo1.** Part 1: Name + age + photos; Part 3: Only names.

      • Photo2. Part 1: Name + age; Part 3: Names+Photos.

      • PhotoEnd. Part 1: Only names; Part 3: Names+Photos.

      • Behavioural information manipulation: We manipulated between participants whether the old man or the young man was the positively portrayed target (there was always one positively-portrayed and one negatively-portrayed target).

  • Measures:

    • Self-reported evaluation
    • IAT
    • Memory test
allds <- read.csv(paste(dir.out, "allds.csv", sep = "\\"))
allok <- read.csv(paste(dir.out, "allok.csv", sep = "\\"))

allok$positive <- ifelse(allok$oldVlnc == "g", "old", ifelse(allok$oldVlnc == "b", "yng", NA))
allok$iat.order2 <- ifelse(allok$yngGd.first & !allok$oldGd.first, "yg.first", ifelse(!allok$yngGd.first & allok$oldGd.first, "og.first", ifelse(allok$yngGd.first & allok$oldGd.first, NA, "error")))

allok$iat.order1 <- ifelse(allok$iat.order2 == "yg.first", ifelse(allok$positive == "yng", "gFirst", ifelse(allok$positive == "old", "bFirst", NA)), ifelse(allok$iat.order2 == "og.first", ifelse(allok$positive == "old", "gFirst", ifelse(allok$positive == "yng", "bFirst", NA)), NA))

allok$condition <- factor(paste(allok$part1Cond, allok$part3Cond, sep = ","))

Exclusion and completion

How many reached the final task (the memory test), out of those who started the study

allds$started <- !is.na(allds$task4)
allds$completed <- !is.na(allds$task10)
my.freq(allds$completed[which(allds$started)])

Of the 1431 participants who started the study, 1093 (76%) completed.

The planned exclusions:

  • We will exclude participants with poor performance in the IAT (above 10% fast trials, Greenwald et al., 2003).

  • We will exclude participants with less than 70% accuracy rate in the last 20 trials of the matching task.

  • Participants who did not respond to all the questions in a specific questionnaire will be excluded from all the analyses that include that questionnaire, but not from other analyses.

IAT performance (0 means ok)

my.freq(allds$SUBEXCL)

38 did not perform the IAT well, but probably some of them did not complete the study, dropping out during the IAT (those with a value of 2). So, let’s see the IAT performance status, only among participants who reached the final task:

my.freq(allds$SUBEXCL[which(allds$completed)])

So, we can say that close to 2% of the participnats (20) performed poorly. This is a bit higher than the usual (which is about 1%).

Accuracy in the matching task:

mysum(allds$match.acc, probs = c(0.05, 0.036))
       n        M       SD       SE      med       5%     3.6% 
1162.000    0.936    0.102    0.002    0.958    0.750    0.667 

People were generally accurate, with less than 4% performing too poorly.

We also excluded participants with data that suggested a technical mistake (the conditions were not saved correctly)

Overall, here are the number of exclusions and inclusions:

pts <- list()
pts$n.started <- sum(allds$started)
pts$n.completed <- sum(allds$completed, na.rm = T)
pts$IAT.poor <- sum(allds$SUBEXCL[allds$completed] > 0, na.rm = T)
pts$not.all.evaluations <- sum(allds$n.atts != 12, na.rm = T)
pts$tech.prob <- sum(allds$conds.dup[allds$completed] | !allds$cond.ok[allds$completed], na.rm = T)
pts$match.too.many.errors <- sum(allds$SUBEXCL[allds$completed] > 0, na.rm = T)
pts$total.ok <- nrow(allok)
pts
$n.started
[1] 1431

$n.completed
[1] 1093

$IAT.poor
[1] 20

$not.all.evaluations
[1] 39

$tech.prob
[1] 89

$match.too.many.errors
[1] 20

$total.ok
[1] 925

We planned to collect data from 1000 participants, so the sample is 7.5% smaller than we hoped for.

Age:

mysum(allok$age)
      n       M      SD      SE     med 
925.000  35.806  15.295   0.503  31.000 

A plot of the age percentiles:

plot.percentile(allok$age, yTitle = "Age", xTitle = "Percentile")

Probably, the 114 years-old lied.

Gender:

my.freq(allok$gender)

Language. We asked: Which of the following is the most accurate? answers: [‘English is my native language’, ‘English is not my native langauge, but it is my primary language’, ‘English is not my native language, and not my primary language, but I am fluent in English’, ‘I understand some English’ ] coded as 1, 2, 3, and 4 (-999 selected a “decline to answer” button at the bottom of the page)

my.freq(allok$lang)

We did not plan to exclude based on this question, but we could explore in the Extended Results section, whether the results change based on this question.

Quick results

IAT

The IAT preference for the positively-portrayed target over the negatively-portrayed target, as a function of the group affiliation of the positively-portrayed target, and the induction and measurement conditions.

Violin plot:

my.violin(DV = allok$IAT.gb, xFactor = allok$condition, fillFactor = allok$positive) + theme(axis.text.x = element_text(size = 11))

(The rectangle is the median, the circle is the mean, and the error-bars are for SE). The difference between the two groups (old vs. yng) is the indication of age-bias. It seems clear only in the name+age,name+photo condition.

Descriptive Statistics:

mysumBy(IAT.gb ~ part1Cond + part3Cond + positive, dt = allok)

Including the IAT’s block order condition (whether the positively portrayed was paired first with Good words or Bad words in the IAT):

knitr::kable(mysumBy(IAT.gb ~ iat.order1 + part1Cond + part3Cond + positive, dt = allok))
var iat.order1 part1Cond part3Cond positive n M SD SE med
IAT.gb bFirst name-only name-only old 50 0.216 0.440 0.062 0.196
IAT.gb bFirst name-only name-only yng 48 0.186 0.355 0.051 0.190
IAT.gb bFirst name-only name+photo old 47 0.181 0.349 0.051 0.215
IAT.gb bFirst name-only name+photo yng 45 0.183 0.325 0.048 0.144
IAT.gb bFirst name+age name-only old 52 0.038 0.362 0.050 0.004
IAT.gb bFirst name+age name-only yng 42 0.171 0.321 0.050 0.139
IAT.gb bFirst name+age name+photo old 47 0.042 0.298 0.044 0.083
IAT.gb bFirst name+age name+photo yng 46 0.308 0.348 0.051 0.321
IAT.gb bFirst name+age+photo name-only old 48 0.274 0.356 0.051 0.208
IAT.gb bFirst name+age+photo name-only yng 48 0.176 0.391 0.056 0.163
IAT.gb gFirst name-only name-only old 39 -0.062 0.392 0.063 -0.080
IAT.gb gFirst name-only name-only yng 43 0.043 0.427 0.065 0.029
IAT.gb gFirst name-only name+photo old 40 -0.109 0.320 0.051 -0.127
IAT.gb gFirst name-only name+photo yng 37 0.092 0.357 0.059 0.103
IAT.gb gFirst name+age name-only old 56 -0.126 0.351 0.047 -0.083
IAT.gb gFirst name+age name-only yng 48 -0.056 0.350 0.051 -0.067
IAT.gb gFirst name+age name+photo old 36 -0.089 0.384 0.064 -0.105
IAT.gb gFirst name+age name+photo yng 53 -0.014 0.396 0.054 0.012
IAT.gb gFirst name+age+photo name-only old 51 0.028 0.458 0.064 -0.095
IAT.gb gFirst name+age+photo name-only yng 49 0.049 0.416 0.059 0.038

There was a clear “reverse” block-order effect. That occurs when the expected score (preference for good) is stronger when that the compatible pairing (in our case, good person + good words) comes at the end, rather than at the beginning of the IAT. The hypothesized reason for such an effect is that the practicing on the sides change of the target categories (in Block 5) is too lonq.

When the negatively-portrayed target individual was paired with Good words in Blocks 3 & 4 of the IAT (i.e., first), there was a pro-young bias only in the conditions [name+age, name-only] & (especially in) [name+age, name+photo]. When the positively-portrayed target individual was paired with Bad words in Blocks 3 & 4 of the IAT (i.e., first), the larger pro-young bias was in the condition [name-only, name+photo]. This inconsistency is not very reassuring.

Let’s see it by the complete condition:

knitr::kable(mysumBy(IAT.gb ~ iat.order1 + positive + condition, dt = allok))
var iat.order1 positive condition n M SD SE med
IAT.gb bFirst old name-only,name-only 50 0.216 0.440 0.062 0.196
IAT.gb bFirst old name-only,name+photo 47 0.181 0.349 0.051 0.215
IAT.gb bFirst old name+age,name-only 52 0.038 0.362 0.050 0.004
IAT.gb bFirst old name+age,name+photo 47 0.042 0.298 0.044 0.083
IAT.gb bFirst old name+age+photo,name-only 48 0.274 0.356 0.051 0.208
IAT.gb bFirst yng name-only,name-only 48 0.186 0.355 0.051 0.190
IAT.gb bFirst yng name-only,name+photo 45 0.183 0.325 0.048 0.144
IAT.gb bFirst yng name+age,name-only 42 0.171 0.321 0.050 0.139
IAT.gb bFirst yng name+age,name+photo 46 0.308 0.348 0.051 0.321
IAT.gb bFirst yng name+age+photo,name-only 48 0.176 0.391 0.056 0.163
IAT.gb gFirst old name-only,name-only 39 -0.062 0.392 0.063 -0.080
IAT.gb gFirst old name-only,name+photo 40 -0.109 0.320 0.051 -0.127
IAT.gb gFirst old name+age,name-only 56 -0.126 0.351 0.047 -0.083
IAT.gb gFirst old name+age,name+photo 36 -0.089 0.384 0.064 -0.105
IAT.gb gFirst old name+age+photo,name-only 51 0.028 0.458 0.064 -0.095
IAT.gb gFirst yng name-only,name-only 43 0.043 0.427 0.065 0.029
IAT.gb gFirst yng name-only,name+photo 37 0.092 0.357 0.059 0.103
IAT.gb gFirst yng name+age,name-only 48 -0.056 0.350 0.051 -0.067
IAT.gb gFirst yng name+age,name+photo 53 -0.014 0.396 0.054 0.012
IAT.gb gFirst yng name+age+photo,name-only 49 0.049 0.416 0.059 0.038
  • When old was the positive target:
    1. The condition that allowed the most pro-positive IAT score, in both IAT order conditions was “name+age+photo,name-only”;
    2. The least pro-old conditions were “name+age,name-only” and “name+age,name+photo” in one IAT order condition (negative+good first), and there was no clear anti-old condition in the the other IAT order condition, but “name+age,name-only” was the worst.
  • When young was the positive target:
    1. The most pro-young condition was “name+age,name+photo” in both order conditions, and the rest were quite similar, especially when the IAT order conditions was negative+good first.
  • So, the consistent result is that the most pro-young condition was “name+age,name+photo” It suggests that the photo is important at the end, but perhaps only when the age information was present in the induction.

Let’s try to see which conditions show a pro-young bias, by looking at the pro-positive IAT score (preference for the positively-portrayed), by IAT-order condition and by positive-target’s group condition.

knitr::kable(mysumBy(IAT.gb ~ condition + iat.order1 + positive, dt = allok))
var condition iat.order1 positive n M SD SE med
IAT.gb name-only,name-only bFirst old 50 0.216 0.440 0.062 0.196
IAT.gb name-only,name-only bFirst yng 48 0.186 0.355 0.051 0.190
IAT.gb name-only,name-only gFirst old 39 -0.062 0.392 0.063 -0.080
IAT.gb name-only,name-only gFirst yng 43 0.043 0.427 0.065 0.029
IAT.gb name-only,name+photo bFirst old 47 0.181 0.349 0.051 0.215
IAT.gb name-only,name+photo bFirst yng 45 0.183 0.325 0.048 0.144
IAT.gb name-only,name+photo gFirst old 40 -0.109 0.320 0.051 -0.127
IAT.gb name-only,name+photo gFirst yng 37 0.092 0.357 0.059 0.103
IAT.gb name+age,name-only bFirst old 52 0.038 0.362 0.050 0.004
IAT.gb name+age,name-only bFirst yng 42 0.171 0.321 0.050 0.139
IAT.gb name+age,name-only gFirst old 56 -0.126 0.351 0.047 -0.083
IAT.gb name+age,name-only gFirst yng 48 -0.056 0.350 0.051 -0.067
IAT.gb name+age,name+photo bFirst old 47 0.042 0.298 0.044 0.083
IAT.gb name+age,name+photo bFirst yng 46 0.308 0.348 0.051 0.321
IAT.gb name+age,name+photo gFirst old 36 -0.089 0.384 0.064 -0.105
IAT.gb name+age,name+photo gFirst yng 53 -0.014 0.396 0.054 0.012
IAT.gb name+age+photo,name-only bFirst old 48 0.274 0.356 0.051 0.208
IAT.gb name+age+photo,name-only bFirst yng 48 0.176 0.391 0.056 0.163
IAT.gb name+age+photo,name-only gFirst old 51 0.028 0.458 0.064 -0.095
IAT.gb name+age+photo,name-only gFirst yng 49 0.049 0.416 0.059 0.038
  • name-only,name-only: If there was a pro-young bias, it was only in one of the IAT order conditions – [positive-good words first]
  • name-only,name+photo: Again, the only pro-young bias was in the the IAT order condition [positive-good words first]
  • name+age,name-only: There was an overall bias, but perhaps stronger in the IAT order condition [positive-bad words first]
  • name+age,name+photo: This time, a much stronger pro-young bias in the IAT order condition [positive-bad words first]
  • name+age+photo,name-only: No pro-young bias, in any IAT order condition.

Statistical analysis

A complete ANOVA:

library(afex)
aov_ez(id = "session_id", dv = "IAT.gb", data = allok, between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))

Only a small effect of bias:

mysumBy(IAT.gb ~ positive, dt = allok)

This effect looks smaller than in previous research (e.g., a beta of .69 in the original study). Notably, the previous research was a [photo-only, photo-only] condition, and without the matching task. Any of these modifications might have reduced the effect of the target’s group affiliation.

The effect of age, within each condition:

aez <- aov_ez(id = "session_id", dv = "IAT.gb", data = allok, between = c("iat.order1", "condition", "positive"))
emm_s.t <- emmeans(aez, pairwise ~ positive | condition)
emm_s.t[[2]]
condition = name-only,name-only:
 contrast  estimate     SE  df t.ratio p.value
 old - yng  -0.0372 0.0559 905  -0.666  0.5056

condition = name-only,name+photo:
 contrast  estimate     SE  df t.ratio p.value
 old - yng  -0.1015 0.0577 905  -1.760  0.0787

condition = name+age,name-only:
 contrast  estimate     SE  df t.ratio p.value
 old - yng  -0.1017 0.0533 905  -1.907  0.0569

condition = name+age,name+photo:
 contrast  estimate     SE  df t.ratio p.value
 old - yng  -0.1708 0.0559 905  -3.056  0.0023

condition = name+age+photo,name-only:
 contrast  estimate     SE  df t.ratio p.value
 old - yng   0.0385 0.0533 905   0.721  0.4711

Results are averaged over the levels of: iat.order1 

The effect was significant only in the name+age,name+photo condition. This might be the most similar to the original method (photo-only, photo-only). It might suggest that it is important to know the group affiliation in the encoding. However, we can’t easily say that there would not be an effect without that, because the [name-only,name+photo] condition might show an effect with a larger sample. Let’s have a look at effects (the estimates) and our main questions. Here are our research questions regarding the IAT:

  • Is there an age bias in the baseline condition? Definitely not. It is not enough to learn about the age after the encoding, and without showing the age in the IAT.

  • A comparison of the group effect between Photo1 and Age1 will estimate the effect of visual cues on encoding. name+age+photo,name-only (Photo1) vs. name+age,name-only [Age1] = Well, Photo1 is the only that did not show the bias numerically (a positive estimate), so we will not see evidence here that showing photos during encoding was important. If anything, we will see evidence of the opposite, which would not be easily explained.

  • A comparison of Photo2 to Age1 will estimate the effect of visual cues on evaluation. name+age,name+photo [Photo2] vs. name+age,name-only [Age1]: The effect was numerically larger in Photo2 than Age1, but probably not significantly so.

  • A comparison of Age1 to baseline will estimate the effect of mere group knowledge on encoding. name+age,name-only [Age1] vs. name-only,name-only [baseline] = Numerically, it seems that knowing about the age during encoding increased bias, but it is obvious that this difference is not significant.

The planned contrasts

I haven’t succeeded so far in testing the planned contrasts within the complete ANOVA model. Therefore, for now, we’ll test them with ANOVAs on each pair of conditions (as planned in the plan document). This is a less powerful test (the MSEis likely to be larger), and it doesn’t allow for easy correction of the p-values.

A comparison of Photo2 to Age1 will estimate the effect of visual cues on evaluation. name+age,name+photo [Photo2] vs. name+age,name-only [Age1]

ccc <- allok[which(allok$condition %in% c("name+age,name+photo", "name+age,name-only")), ]
mysumBy(IAT.gb ~ condition + positive, dt = ccc)
aov_ez(id = "session_id", dv = "IAT.gb", data = ccc, between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))

No interaction between condition and the group affiliation of the positively-portrayed target individual on the IAT preference for the positively-portrayed individual over the negatively-portrayed individual. With such a small effect, it is unlikely that we will see a significant effect with any alternative analysis.

A comparison of Age1 to baseline will estimate the effect of mere group knowledge on encoding. name+age,name-only [Age1] vs. name-only,name-only [baseline]

ccc <- allok[which(allok$condition %in% c("name-only,name-only", "name+age,name-only")), ]
mysumBy(IAT.gb ~ condition + positive, dt = ccc)
aov_ez(id = "session_id", dv = "IAT.gb", data = ccc, between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))

No interaction between condition and the group affiliation of the positively-portrayed target individual on the IAT preference for the positively-portrayed individual over the negatively-portrayed. Again, the partial eta square seems too small to amount to a significant difference in any alternative statistical analysis.

A comparison of the group effect between Photo1 and Age1 will estimate the effect of visual cues on encoding. name+age+photo,name-only (Photo1) vs. name+age,name-only [Age1]

ccc <- allok[which(allok$condition %in% c("name+age+photo,name-only", "name+age,name-only")), ]
mysumBy(IAT.gb ~ condition + positive, dt = ccc)
aov_ez(id = "session_id", dv = "IAT.gb", data = ccc, between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))

No interaction between condition and the group affiliation of the positively-portrayed target individual on the IAT preference for the positively-portrayed individual over the negatively-portrayed. This time, perhaps a more powerful statistical analysis would result with a significant difference, although the effect is still very small. Importantly, the difference is odd (stronger bias with the photos of the target individuals), which further reduces the confidence in these results.

Self-reported evaluation

Regarding each of the two target individuals, participants responded to 6 questions in the following format: Does the word ATTRIBUTE characterize NAME? In name+photo conditions, the photo of the individual was shown with each question. I will explore these questions further in the “Extended Results” section, but now let’s dive in to the preference score computed from these evaluation questions. We will look at the preference for the positively-portrayed target individual.

my.violin(DV = allok$eval.gb, xFactor = allok$condition, fillFactor = allok$positive) + theme(axis.text.x = element_text(size = 11))

mysumBy(eval.gb ~ part1Cond + part3Cond + positive, dt = allok)

There was always a preference for the positively-portrayed target. In two of the conditions, there seems to be a pro-young bias: in the baseline condition [name-only, name-only] and in the Photo1 [name+age+photo, name-only] conditions. That is unexpected.

Let’s ANOVA in a 5 (condition) x 2 (positive target) ANOVA

aov_ez(id = "session_id", dv = "eval.gb", data = allok, between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

Well, there is no interaction, only a tiny effect of the group of the positively-portray target individual.

mysumBy(eval.gb ~ positive, dt = allok)

A slight pro-young preference.

Still, let’s see the effect of the targets’ group within each condition:

aez <- aov_ez(id = "session_id", dv = "eval.gb", data = allok, between = c("condition", "positive"))
emm_s.t <- emmeans(aez, pairwise ~ positive | condition)
emm_s.t[[2]]
condition = name-only,name-only:
 contrast  estimate   SE  df t.ratio p.value
 old - yng  -16.626 7.01 915  -2.372  0.0179

condition = name-only,name+photo:
 contrast  estimate   SE  df t.ratio p.value
 old - yng    2.606 7.24 915   0.360  0.7188

condition = name+age,name-only:
 contrast  estimate   SE  df t.ratio p.value
 old - yng   -3.336 6.71 915  -0.497  0.6192

condition = name+age,name+photo:
 contrast  estimate   SE  df t.ratio p.value
 old - yng   -0.819 7.00 915  -0.117  0.9069

condition = name+age+photo,name-only:
 contrast  estimate   SE  df t.ratio p.value
 old - yng  -20.352 6.72 915  -3.031  0.0025

Let’s also see those effect in a t-test, mostly to see the Cohen’s d and Bayes Factor: In name+age+photo,name-only

ccc <- allok[which(allok$condition == "name+age+photo,name-only"), ]
ttestIS(data = ccc, vars = c("eval.gb"), group = "positive", bf = T, desc = T, effectSize = T)

 INDEPENDENT SAMPLES T-TEST

 Independent Samples T-Test                                                                                      
 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
                                Statistic    error %         df          p                         Effect Size   
 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
   eval.gb    Student's t       -3.431114                    194.0000    0.0007345    Cohen's d     -0.4901846   
              Bayes factor₁₀     34.17799    7.992516e-10                                                        
 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
   Note. Hₐ μ <sub>old</sub> ≠ μ <sub>yng</sub>


 Group Descriptives                                                         
 ────────────────────────────────────────────────────────────────────────── 
              Group    N     Mean        Median      SD          SE         
 ────────────────────────────────────────────────────────────────────────── 
   eval.gb    old      99    21.74074    10.66667    37.50179    3.769072   
              yng      97    42.09278    33.33333    45.25380    4.594828   
 ────────────────────────────────────────────────────────────────────────── 

In name-only,name-only (the baseline condition)

ccc <- allok[which(allok$condition == "name-only,name-only"), ]
ttestIS(data = ccc, vars = c("eval.gb"), group = "positive", bf = T, desc = T, effectSize = T)

 INDEPENDENT SAMPLES T-TEST

 Independent Samples T-Test                                                                                     
 ────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
                                Statistic    error %        df          p                         Effect Size   
 ────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
   eval.gb    Student's t       -2.221616                   178.0000    0.0275698    Cohen's d     -0.3311995   
              Bayes factor₁₀     1.575094    1.184092e-4                                                        
 ────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
   Note. Hₐ μ <sub>old</sub> ≠ μ <sub>yng</sub>


 Group Descriptives                                                          
 ─────────────────────────────────────────────────────────────────────────── 
              Group    N     Mean        Median       SD          SE         
 ─────────────────────────────────────────────────────────────────────────── 
   eval.gb    old      89    15.02996    0.6666667    51.19278    5.426424   
              yng      91    31.65568     17.00000    49.20688    5.158283   
 ─────────────────────────────────────────────────────────────────────────── 

So, two of the conditions showed small-to-medium pro-young bias in the self-reported evaluation. What’s odd is that these conditions did not show that pro-young bias in the IAT. This might suggest that the lack of pro-young IAT bias in the name-only IATs reflect worst measurement rather than less bias. That is, perhaps IATs with names only are less reliable/valid in attitude formation studies (because the name was just learned). We could test that speculation by looking at the effect of condition on internal consistency. We might also learn something by looking at correlations between IAT and self-report and at discrepancies, as a function of the condition.

Correlations and discrepancies

Correlations

Overall correlation between the evaluation scores. The IAT.prf and eval.diff are preferences for the young target over the old target.

my.htmlTable(cornp(allok[, c("IAT.gb", "eval.gb", "age", "IAT.prf", "eval.diff")]))
varName____ IAT.gb____ eval.gb____ age____ IAT.prf____
1 eval.gb 0.171
< .001
925
2 age -0.013
0.698
925
-0.036
0.273
925
3 IAT.prf -0.021
0.52
925
-0.039
0.237
925
0.007
0.831
925
4 eval.diff -0.008
0.816
925
0.001
0.974
925
0.021
0.523
925
0.237
< .001
925

It might seem odd that the IAT.prf/eval.diff correlation is not the same as the IAT.gb/eval.gb correlation, but I verified (in the sanity checks below) that there is no error here. My guess is that that it can happen if the SDs are changed due to the recoding. In any case, the correlations here are not large, but are in the range of the typical IAT/self-report correlation which is about 0.15-0.70, depending on the attitude object.

Let’s see whether the correlation changed drastically between conditions:

my.htmlTable(corby(inData = allok, xs = c("IAT.gb", "IAT.prf"), ys = c("eval.gb", "eval.diff"), by = c("condition")))
condition____ varName____ eval.gb____ eval.diff____
1 name-only,name-only IAT.gb 0.19
0.011
180
-0.121
0.106
180
2 name-only,name-only IAT.prf -0.1
0.182
180
0.27
< .001
180
3 name-only,name+photo IAT.gb 0.227
0.003
169
0.087
0.261
169
4 name-only,name+photo IAT.prf 0.024
0.753
169
0.316
< .001
169
5 name+age,name-only IAT.gb 0.004
0.957
198
-0.027
0.708
198
6 name+age,name-only IAT.prf -0.109
0.125
198
0.001
0.989
198
7 name+age,name+photo IAT.gb 0.202
0.006
182
0.063
0.396
182
8 name+age,name+photo IAT.prf -0.03
0.691
182
0.252
< .001
182
9 name+age+photo,name-only IAT.gb 0.216
0.002
196
-0.055
0.446
196
10 name+age+photo,name-only IAT.prf 0.037
0.602
196
0.358
< .001
196

The correlation were relatively consistent, excluding the condition ‘name+age,name-only’ which showed zero correlation. I don’t see anything unordinary in that condition, in terms of discrepancy.

Discrepancies

Let’s see effect sizes of the pro-young bias, by condition and measure.

effect_sizes <- by(allok, allok$condition, function(x) {
    d_IAT <- cohen.d(x$IAT.gb[x$positive == "yng"], x$IAT.gb[x$positive == "old"])
    d_eval <- cohen.d(x$eval.gb[x$positive == "yng"], x$eval.gb[x$positive == "old"])
    m_IAT.yng <- mean(x$IAT.gb[x$positive == "yng"], na.rm = T)
    m_IAT.old <- mean(x$IAT.gb[x$positive == "old"], na.rm = T)
    m_eval.yng <- mean(x$eval.gb[x$positive == "yng"], na.rm = T)
    m_eval.old <- mean(x$eval.gb[x$positive == "old"], na.rm = T)
    c(IAT.d = d_IAT$estimate, IAT.ci = d_IAT$conf.int, eval.d = d_eval$estimate, eval.ci = d_eval$conf.int, IAT.m.yng = m_IAT.yng, IAT.m.old = m_IAT.old, eval.m.yng = m_eval.yng, eval.m.old = m_eval.old)
})

effect_sizes_df <- as.data.frame(round(do.call(rbind, effect_sizes), 2))
effect_sizes_df$IAT.ci <- paste0("[", effect_sizes_df$IAT.ci.lower, " - ", effect_sizes_df$IAT.ci.upper, "]")
effect_sizes_df$eval.ci <- paste0("[", effect_sizes_df$eval.ci.lower, " - ", effect_sizes_df$eval.ci.upper, "]")
effect_sizes_df$IAT.ci.lower <- NULL
effect_sizes_df$IAT.ci.upper <- NULL
effect_sizes_df$eval.ci.lower <- NULL
effect_sizes_df$eval.ci.upper <- NULL
effect_sizes_df[, c("IAT.d", "IAT.ci", "eval.d", "eval.ci", "IAT.m.yng", "IAT.m.old", "eval.m.yng", "eval.m.old")]

There is an overlap in the CI of the effect-size of the self-report (eval) and the IAT score in all the conditions. The only possible exception is Photo1 [name+age+photo,name-only], with the largest pro-young bias in the self-reported evaluation and the smallest (indeed, pro-old) bias in the IAT. Because this is unexpected and not easily explained, I suspect this is a statistical fluke, until replication.

Memory

The memory question was: Who appeared with the following behavior?
<%=questionsData.bv%>

Response options were either the names or the photos of the two targets, and “None of them”.

We presented 4 positive and 4 negative behaviors of each target, and 4 positive and 4 negative novel behaviors (total of 24 behaviors).

Accuracy

Memory accuracy by type of question (who = who was really characterized by this question [y = young, o = old, n = none], vlnc = whether the behavior was positive or negative [p or n]):

mmm <- mymelt(dt = allok, formula = macc.o.neg + macc.o.pos + macc.y.neg + macc.y.pos + macc.n.neg + macc.n.pos ~ session_id + condition + positive)
mmm$who <- substr(mmm$variable, 6, 6)
mmm$vlnc <- substr(mmm$variable, 8, 8)
mysumBy(value ~ who + vlnc, dt = mmm)

Negative behaviors, behaviors performed by the young target, and behaviors that appeared in the induction were remembered better.

With the positively-portrayed group affiliation condition:

knitr::kable(mysumBy(value ~ vlnc + positive + who, dt = mmm))
var vlnc positive who n M SD SE med
value n old n 451 0.729 0.356 0.016 1.00
value n old o 451 0.774 0.289 0.013 1.00
value n old y 451 0.835 0.246 0.011 1.00
value n yng n 454 0.743 0.343 0.016 1.00
value n yng o 454 0.792 0.262 0.012 1.00
value n yng y 454 0.849 0.255 0.012 1.00
value p old n 451 0.684 0.358 0.017 0.75
value p old o 451 0.713 0.280 0.013 0.75
value p old y 451 0.749 0.263 0.012 0.75
value p yng n 454 0.702 0.350 0.016 0.75
value p yng o 454 0.730 0.281 0.013 0.75
value p yng y 454 0.777 0.250 0.012 0.75

Regarding negative behaviors:

  • when the positive was the old target, people had better memory for congruent behaviors (negative by young) than for incongruent behaviors (negative by old).
  • When the positive was the young target, people had better memory for incongruent behaviors (negative by young) than for incongruent behaviors (negative by old).

Regarding positive behaviors:

  • When the positive was the old target, people had better memory for incongruent behaviors (positive by young) than for congruent behaviors (positive by old).
  • When the positive was the young target, people had better memory for congruent behaviors (positive by young) than for incongruent behaviors (positive by old).

So, it simply seems that people had better memory for the young than for the old target, and not sure whether there was any moderation by congruency between the behavior and the majority of behaviors performed by the target.

Let’s look at the same results with a different order, to see whether the positive target group affiliation conditions had any effect:

knitr::kable(mysumBy(value ~ who + positive + vlnc, dt = mmm))
var who positive vlnc n M SD SE med
value n old n 451 0.729 0.356 0.016 1.00
value n old p 451 0.684 0.358 0.017 0.75
value n yng n 454 0.743 0.343 0.016 1.00
value n yng p 454 0.702 0.350 0.016 0.75
value o old n 451 0.774 0.289 0.013 1.00
value o old p 451 0.713 0.280 0.013 0.75
value o yng n 454 0.792 0.262 0.012 1.00
value o yng p 454 0.730 0.281 0.013 0.75
value y old n 451 0.835 0.246 0.011 1.00
value y old p 451 0.749 0.263 0.012 0.75
value y yng n 454 0.849 0.255 0.012 1.00
value y yng p 454 0.777 0.250 0.012 0.75

Still hard to say.

We could get the conditions in, but probably best to see that in a plot. The top heading indicates the group of the postivie target. The next heading indicates the condition. The fill is the valence of the behavior. The x-axis is who was characterized by this behavior in the induction.

my.violin(DV = mmm$value, xFactor = mmm$who, fillFactor = mmm$vlnc, facet1 = mmm$positive, facet2 = mmm$condition)

The patterns might be different sometimes, but it’s still hard to tell. So, perhaps we need an ANOVA to get a better idea about effects.

aov_ez(id = "session_id", dv = "value", data = mmm, within = c("who", "vlnc"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

The main effect of condition:

mysumBy(value ~ condition, dt = mmm)

Better memory when more individuating details were provided about the target in the induction. Providing the photo in the memory test doesn’t seem to improve memory.

Comparing all levels:

m1 <- emmeans(aaa, ~condition)
pairs(m1)
 contrast                                            estimate     SE  df t.ratio p.value
 (name-only,name-only) - (name-only,name+photo)        0.0626 0.0203 895   3.091  0.0175
 (name-only,name-only) - (name+age,name-only)         -0.0695 0.0194 895  -3.579  0.0033
 (name-only,name-only) - (name+age,name+photo)        -0.0503 0.0198 895  -2.539  0.0831
 (name-only,name-only) - (name+age+photo,name-only)   -0.1057 0.0194 895  -5.436  <.0001
 (name-only,name+photo) - (name+age,name-only)        -0.1321 0.0198 895  -6.669  <.0001
 (name-only,name+photo) - (name+age,name+photo)       -0.1129 0.0202 895  -5.588  <.0001
 (name-only,name+photo) - (name+age+photo,name-only)  -0.1684 0.0198 895  -8.481  <.0001
 (name+age,name-only) - (name+age,name+photo)          0.0191 0.0194 895   0.989  0.8605
 (name+age,name-only) - (name+age+photo,name-only)    -0.0363 0.0190 895  -1.911  0.3119
 (name+age,name+photo) - (name+age+photo,name-only)   -0.0554 0.0194 895  -2.856  0.0355

Results are averaged over the levels of: positive, vlnc, who 
P value adjustment: tukey method for comparing a family of 5 estimates 

Most of them significant.

Next, we found a main effect of the person who was actually characterized by the behavior:

mysumBy(value ~ who, dt = mmm)

Better for young targets.

Next, a main effect of valence of the behavior:

mysumBy(value ~ vlnc, dt = mmm)

More accurate for negative behaviors.

Next, the induction condition moderated the effect of who was actually characterized by the behavior:

knitr::kable(mysumBy(value ~ condition + who, dt = mmm))
var condition who n M SD SE med
value name-only,name-only n 352 0.754 0.325 0.017 1.00
value name-only,name-only o 352 0.658 0.311 0.016 0.75
value name-only,name-only y 352 0.750 0.294 0.016 0.75
value name-only,name+photo n 326 0.613 0.398 0.022 0.75
value name-only,name+photo o 326 0.632 0.309 0.017 0.75
value name-only,name+photo y 326 0.729 0.277 0.015 0.75
value name+age,name-only n 390 0.749 0.333 0.017 1.00
value name+age,name-only o 390 0.799 0.255 0.013 1.00
value name+age,name-only y 390 0.822 0.247 0.012 1.00
value name+age,name+photo n 358 0.690 0.365 0.019 0.75
value name+age,name+photo o 358 0.798 0.231 0.012 0.75
value name+age,name+photo y 358 0.825 0.229 0.012 1.00
value name+age+photo,name-only n 384 0.753 0.322 0.016 1.00
value name+age+photo,name-only o 384 0.852 0.223 0.011 1.00
value name+age+photo,name-only y 384 0.873 0.207 0.010 1.00

No effects at all for the group affiliation of the positively portrayed target. People almost always had better memory for the young target’s behaviors, but memory for the old person’s behavior was not always better than for novel behaviors. Of particular note, in the [name-only, name-only] condition, the memory for the old target’s behavior was the worst, and the memory for the young target’s behaviors was not better than memory for the novel behaviors (of course, that could result from a bias to answer “none”, in that condition).

There were a couple of very small (but significant) effect. One was a three-way interaction between condition, who performed the behavior, and the behavior valence.

Let’s plot that interaction within each condition, to see where the difference is:

my.violin(DV = mmm$value, xFactor = mmm$who, fillFactor = mmm$vlnc, facet1 = mmm$condition)

I’m not sure. Perhaps if plot a bit differently:

my.violin(DV = mmm$value, xFactor = mmm$vlnc, fillFactor = mmm$who, facet1 = mmm$condition)

The patterns are different, but I don’t find a clear way to describe anything interesting about it.

knitr::kable(mysumBy(value ~ condition + who + vlnc, dt = mmm))
var condition who vlnc n M SD SE med
value name-only,name-only n n 176 0.780 0.313 0.023 1.00
value name-only,name-only n p 176 0.727 0.336 0.025 0.75
value name-only,name-only o n 176 0.723 0.315 0.023 0.75
value name-only,name-only o p 176 0.594 0.294 0.022 0.75
value name-only,name-only y n 176 0.776 0.297 0.022 1.00
value name-only,name-only y p 176 0.724 0.290 0.022 0.75
value name-only,name+photo n n 163 0.633 0.407 0.031 0.75
value name-only,name+photo n p 163 0.592 0.389 0.030 0.75
value name-only,name+photo o n 163 0.653 0.324 0.025 0.75
value name-only,name+photo o p 163 0.610 0.293 0.023 0.75
value name-only,name+photo y n 163 0.775 0.270 0.021 0.75
value name-only,name+photo y p 163 0.683 0.278 0.021 0.75
value name+age,name-only n n 195 0.781 0.322 0.023 1.00
value name+age,name-only n p 195 0.718 0.342 0.024 0.75
value name+age,name-only o n 195 0.833 0.243 0.017 1.00
value name+age,name-only o p 195 0.764 0.263 0.019 0.75
value name+age,name-only y n 195 0.842 0.257 0.018 1.00
value name+age,name-only y p 195 0.801 0.235 0.017 0.75
value name+age,name+photo n n 179 0.712 0.365 0.027 1.00
value name+age,name+photo n p 179 0.668 0.365 0.027 0.75
value name+age,name+photo o n 179 0.813 0.228 0.017 0.75
value name+age,name+photo o p 179 0.784 0.233 0.017 0.75
value name+age,name+photo y n 179 0.885 0.200 0.015 1.00
value name+age,name+photo y p 179 0.764 0.240 0.018 0.75
value name+age+photo,name-only n n 192 0.760 0.323 0.023 1.00
value name+age+photo,name-only n p 192 0.746 0.322 0.023 0.75
value name+age+photo,name-only o n 192 0.871 0.206 0.015 1.00
value name+age+photo,name-only o p 192 0.832 0.237 0.017 1.00
value name+age+photo,name-only y n 192 0.918 0.187 0.013 1.00
value name+age+photo,name-only y p 192 0.828 0.217 0.015 1.00

Well, who knows. We could look at the whoXvalence interaction within each of the five conditions.

unique(mmm$condition)
[1] name+age,name+photo      name+age,name-only       name+age+photo,name-only name-only,name+photo     name-only,name-only     
Levels: name-only,name-only name-only,name+photo name+age,name-only name+age,name+photo name+age+photo,name-only

Memory accuracy by confidence

The confidence question was: How certain are you about this answer?‘, response options: ’guess’, ‘probably’, ‘certain’ (recoded as 1, 2, 3)

mcm <- mymelt(dt = allok, formula = conf.n.neg.1 + conf.n.neg.2 + conf.n.neg.3 + conf.n.pos.1 + conf.n.pos.2 + conf.n.pos.3 + conf.o.neg.1 + conf.o.neg.2 + conf.o.neg.3 + conf.o.pos.1 + conf.o.pos.2 + conf.o.pos.3 + conf.y.neg.1 + conf.y.neg.2 + conf.y.neg.3 + conf.y.pos.1 + conf.y.pos.2 + conf.y.pos.3 ~ session_id + condition + positive)

mcm$who <- substr(mcm$variable, 6, 6)
mcm$vlnc <- substr(mcm$variable, 8, 8)
mcm$conf <- substr(mcm$variable, 12, 12)

mcm <- mcm[complete.cases(mcm), ]

Memory accuracy by confidence:

mysumBy(value ~ conf, dt = mcm)

As can be expected, more accuracy when people are more confident about their response.

Memory accuracy by response

It is possible that a bias to respond “young” caused the higher accuracy rate regarding the young target. To examine that possibility, we will start by examining the number of responses.

Number of memory responses

mmm <- mymelt(formula = m.n.byresp.non.neg + m.n.byresp.non.pos + m.n.byresp.yng.neg + m.n.byresp.yng.pos + m.n.byresp.old.neg + m.n.byresp.old.pos ~ session_id + condition + positive, dt = allok)
mmm$resp <- substr(mmm$variable, 12, 14)
mmm$vlnc <- substr(mmm$variable, 16, 16)
mmm$n.resps <- ifelse(is.na(mmm$value), 0, mmm$value)
mysumBy(n.resps ~ resp + vlnc, dt = mmm)

We see a clear bias toward responding “yng” and against responding “old”.

Plot:

my.violin(DV = mmm$n.resps, xFactor = mmm$vlnc, fillFactor = mmm$resp)

The young-response-bias effect seems larger for negative than for positive behaviors.

Let’s see this also with positive-target group:

my.violin(DV = mmm$n.resps, xFactor = mmm$vlnc, fillFactor = mmm$resp, facet1 = mmm$positive)

When the positively-portrayed target individuals was the old man, the young-response-bias was stronger for negative than for positive behaviors. When the positively-portrayed target individuals was the young man, the young-response-bias was stronger for positive than for negative behaviors. That means that people used the consistency between the target’s valence and the behavior valence as a memory-cue. That makese sense, of course.

Let’s ANOVA this with the other manipulated factors:

aov_ez(id = "session_id", dv = "n.resps", data = mmm, within = c("resp", "vlnc"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

A strong effect of response:

mysumBy(n.resps ~ resp, dt = mmm)

A small effect for the three-way interaction between response, positively-protrayed target’s group, and behavior valence. It is the valence compatibilty effect we noticed earlier:

knitr::kable(mysumBy(n.resps ~ resp + positive + vlnc, dt = mmm))
var resp positive vlnc n M SD SE med
n.resps non old n 466 3.124 1.676 0.078 4
n.resps non old p 466 3.251 1.880 0.087 4
n.resps non yng n 459 3.246 1.526 0.071 4
n.resps non yng p 459 3.279 1.744 0.081 4
n.resps old old n 466 3.764 1.491 0.069 4
n.resps old old p 466 4.159 1.866 0.086 4
n.resps old yng n 459 4.155 1.543 0.072 4
n.resps old yng p 459 3.908 1.551 0.072 4
n.resps yng old n 466 4.627 1.860 0.086 4
n.resps yng old p 466 4.116 1.711 0.079 4
n.resps yng yng n 459 4.403 1.549 0.072 4
n.resps yng yng p 459 4.632 1.811 0.085 4

A small interaction between the condition and the response:

knitr::kable(mysumBy(n.resps ~ condition + resp, dt = mmm))
var condition resp n M SD SE med
n.resps name-only,name-only non 360 3.472 1.620 0.085 4
n.resps name-only,name-only old 360 3.714 1.810 0.095 4
n.resps name-only,name-only yng 360 4.492 1.929 0.102 4
n.resps name-only,name+photo non 338 2.754 1.923 0.105 3
n.resps name-only,name+photo old 338 3.920 1.784 0.097 4
n.resps name-only,name+photo yng 338 4.678 2.035 0.111 4
n.resps name+age,name-only non 396 3.384 1.690 0.085 4
n.resps name+age,name-only old 396 4.053 1.475 0.074 4
n.resps name+age,name-only yng 396 4.351 1.588 0.080 4
n.resps name+age,name+photo non 364 3.096 1.658 0.087 4
n.resps name+age,name+photo old 364 4.212 1.682 0.088 4
n.resps name+age,name+photo yng 364 4.462 1.724 0.090 4
n.resps name+age+photo,name-only non 392 3.362 1.585 0.080 4
n.resps name+age+photo,name-only old 392 4.064 1.349 0.068 4
n.resps name+age+photo,name-only yng 392 4.276 1.443 0.073 4

When photos appear in the memory test, people were less likely to choose “none” than in the other conditions.

This 2-way interaction was further moderated by the behavior valence:

knitr::kable(mysumBy(n.resps ~ vlnc + resp + condition, dt = mmm))
var vlnc resp condition n M SD SE med
n.resps n non name-only,name-only 180 3.394 1.470 0.110 4
n.resps n non name-only,name+photo 169 2.716 1.787 0.137 3
n.resps n non name+age,name-only 198 3.449 1.639 0.116 4
n.resps n non name+age,name+photo 182 3.055 1.551 0.115 4
n.resps n non name+age+photo,name-only 196 3.250 1.479 0.106 4
n.resps n old name-only,name-only 180 3.928 1.834 0.137 4
n.resps n old name-only,name+photo 169 3.722 1.680 0.129 4
n.resps n old name+age,name-only 198 4.056 1.307 0.093 4
n.resps n old name+age,name+photo 182 4.011 1.534 0.114 4
n.resps n old name+age+photo,name-only 196 4.041 1.256 0.090 4
n.resps n yng name-only,name-only 180 4.350 1.754 0.131 4
n.resps n yng name-only,name+photo 169 4.917 2.057 0.158 5
n.resps n yng name+age,name-only 198 4.273 1.569 0.112 4
n.resps n yng name+age,name+photo 182 4.687 1.739 0.129 4
n.resps n yng name+age+photo,name-only 196 4.408 1.384 0.099 4
n.resps p non name-only,name-only 180 3.550 1.757 0.131 4
n.resps p non name-only,name+photo 169 2.793 2.055 0.158 3
n.resps p non name+age,name-only 198 3.318 1.741 0.124 4
n.resps p non name+age,name+photo 182 3.137 1.761 0.131 4
n.resps p non name+age+photo,name-only 196 3.474 1.681 0.120 4
n.resps p old name-only,name-only 180 3.500 1.764 0.131 4
n.resps p old name-only,name+photo 169 4.118 1.867 0.144 4
n.resps p old name+age,name-only 198 4.051 1.630 0.116 4
n.resps p old name+age,name+photo 182 4.412 1.799 0.133 4
n.resps p old name+age+photo,name-only 196 4.087 1.439 0.103 4
n.resps p yng name-only,name-only 180 4.633 2.085 0.155 4
n.resps p yng name-only,name+photo 169 4.438 1.990 0.153 4
n.resps p yng name+age,name-only 198 4.429 1.607 0.114 4
n.resps p yng name+age,name+photo 182 4.236 1.683 0.125 4
n.resps p yng name+age+photo,name-only 196 4.143 1.492 0.107 4

For negative behaviors, the “none” response was less likely and the “young” response was more likely when photos appeared in the memory test. For positive behaviors, the “none” response was less likely and the “old” response was more likely when photos appeared in the memory test. I am not sure what that might mean.

Let’s see separate ANOVAs. For negative behaviors only:

aov_ez(id = "session_id", dv = "n.resps", data = mmm[which(mmm$vlnc == "n"), ], within = c("resp"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

for positive behavior only:

aov_ez(id = "session_id", dv = "n.resps", data = mmm[which(mmm$vlnc == "p"), ], within = c("resp"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

The conditionXresp interaction effects are of the same size, but we saw they were different when we unpacked the 3-way interaction above.

mysumBy(n.resps ~ vlnc + resp, dt = mmm)

So, all and all, it could be that people’s tendency to respond “young” caused the higher accuracy rates for behavior of the young target than behaviors of the old target (and for novel behaviors). But, there’s another way to test that, nex…

Memory accuracy by response

Let’s see whether there are differences in memory accuracy between different responses (rather than betwen behaviors of the young target vs. old target vs. novel behaviors)

mmm <- mymelt(formula = m.acc.byresp.non.neg + m.acc.byresp.non.pos + m.acc.byresp.yng.neg + m.acc.byresp.yng.pos + m.acc.byresp.old.neg + m.acc.byresp.old.pos ~ session_id + condition + positive, dt = allok)
mmm$resp <- substr(mmm$variable, 14, 16)
mmm$vlnc <- substr(mmm$variable, 18, 18)
mysumBy(value ~ resp + vlnc, dt = mmm)

Indeed, we no longer see better accuracy for the young response. Rather, the best accuracy is for novel behaviors. This is the mirror image of the response frequency. It makes some sense because

DOES IT MEAN WE NEED TO CONDUCT AN SDT ANALYSIS OF SOME SORTS?

Extended results

Internal consistencies

alphas <- list()

alphas$self.report.negative.attributes.old <- round(psych::alpha(allok[, c("eval.oneg1", "eval.oneg2", "eval.oneg3")])$total$std.alpha, 2)
alphas$self.report.negative.attributes.young <- round(psych::alpha(allok[, c("eval.yneg1", "eval.yneg2", "eval.yneg3")])$total$std.alpha, 2)
alphas$self.report.positive.attributes.old <- round(psych::alpha(allok[, c("eval.opos1", "eval.opos2", "eval.opos3")])$total$std.alpha, 2)
alphas$self.report.positive.attributes.young <- round(psych::alpha(allok[, c("eval.ypos1", "eval.ypos2", "eval.ypos3")])$total$std.alpha, 2)
alphas$IAT.alpha <- round(psych::alpha(allok[, c("IAT1", "IAT2")])$total$std.alpha, 2)

alphas
$self.report.negative.attributes.old
[1] 0.82

$self.report.negative.attributes.young
[1] 0.79

$self.report.positive.attributes.old
[1] 0.87

$self.report.positive.attributes.young
[1] 0.88

$IAT.alpha
[1] 0.6

Quite typical for these evaluation measures.

In the quick results section, we used the variable eval.diff which was computed as the difference of differences

allok$posTarget <- ifelse(allok$positive == "yng", 1, ifelse(allok$positive == "old", 0, NA))

So, there are a few self-reported evaluation scores here, and we can examine the correlation between them [posTarget is 1 if young, 0 if old; IAT.prf is the IAT score recoded to reflect a preference for the young individual over the old individual]:

my.htmlTable(cornp(allok[, c("ypos", "yneg", "opos", "oneg", "IAT.prf", "posTarget")]))
varName____ ypos____ yneg____ opos____ oneg____ IAT.prf____
1 yneg -0.336
< .001
925
2 opos 0.047
0.153
925
0.344
< .001
925
3 oneg 0.335
< .001
925
0.162
< .001
925
-0.381
< .001
925
4 IAT.prf 0.151
< .001
925
-0.107
0.001
925
-0.197
< .001
925
0.144
< .001
925
5 posTarget 0.352
< .001
925
-0.23
< .001
925
-0.333
< .001
925
0.314
< .001
925
0.197
< .001
925

ok. These results might mean that the behaviors (i.e., the attitude induction) had a weaker influence on the IAT score and the negative-attributes ratings of the young target than on the other evaluation measures (the differences are significant).

Results by attribute questions valence

We’ll analyze the self-report evaluation with an attribute-valence factor (rather than the difference score). Because these are many factors, it will be easier to start with the ANOVA, and use that for guiding our attention to effects.

mmm <- mymelt(formula = ypos + yneg + opos + oneg ~ session_id + condition + positive, dt = allok)
mmm$group <- substr(mmm$variable, 1, 1)
mmm$att.vlnc <- substr(mmm$variable, 2, 5)
rrr <- aov_ez(id = "session_id", dv = "value", data = mmm, within = c("group", "att.vlnc"), between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))
print(rrr)
Anova Table (Type 3 tests)

Response: value
                              Effect     df    MSE          F   pes p.value
1                          condition 4, 915 498.73       0.10 <.001    .983
2                           positive 1, 915 498.73       2.60  .003    .107
3                 condition:positive 4, 915 498.73       0.12 <.001    .975
4                              group 1, 915  87.85       1.53  .002    .216
5                    condition:group 4, 915  87.85       0.42  .002    .793
6                     positive:group 1, 915  87.85  19.52 ***  .021   <.001
7           condition:positive:group 4, 915  87.85       1.31  .006    .263
8                           att.vlnc 1, 915 509.41  22.28 ***  .024   <.001
9                 condition:att.vlnc 4, 915 509.41       0.93  .004    .445
10                 positive:att.vlnc 1, 915 509.41       1.42  .002    .234
11       condition:positive:att.vlnc 4, 915 509.41     2.61 *  .011    .034
12                    group:att.vlnc 1, 915 552.42     6.17 *  .007    .013
13          condition:group:att.vlnc 4, 915 552.42     2.15 +  .009    .073
14           positive:group:att.vlnc 1, 915 552.42 283.82 ***  .237   <.001
15 condition:positive:group:att.vlnc 4, 915 552.42       0.98  .004    .417
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1

positive = the group of the positively-portrayed target individual; group = the group of the evaluated target individual (i.e., whether participants the evaluation pertained to the young person or the old person); att.vlnc = whether the evaluation was for positive attributes or negative attributes.;

The largest effect is a 3-way interaction between the group of the positively-portrayed target, the evaluated group and the valence of the evaluation attributes. It probably simply reflects the effect of the induction: The relevant means:

mysumBy(value ~ positive + group + att.vlnc, dt = mmm)

Yes, that is the induction effect: when the positively-portrayed target was old (positive=old), participants reported that positive attributes characterize the old target more than negative attributes and the opposite for the young target. The pattern was the complete opposite when the positively-portrayed target was young (positive=yng).

The next effect was a moderate main effect of attribute valence:

mysumBy(value ~ att.vlnc, dt = mmm)

Participants thought the positive attributes chracterized the target individuals more than negative attributes. A positivity-bias.

Next, an interaction between the group of the evaluated individual (“group”) and the group of positively-portrayed target (“positive”)

mysumBy(value ~ positive + group, dt = mmm)

Well, because the attribute valence is not involved in this effect, this is just an odd effect reflecting higher rating (regradless of attribute valence) in the positive=young than in the positive=old condition.

Next, a small 3-way interaction of condition by positive by att.vlnc:

knitr::kable(mysumBy(value ~ condition + positive + att.vlnc, dt = mmm))
var condition positive att.vlnc n M SD SE med
value name-only,name-only old neg 178 37.934 22.927 1.718 34.167
value name-only,name-only old pos 178 43.408 22.642 1.697 45.000
value name-only,name-only yng neg 182 42.332 22.505 1.668 43.333
value name-only,name-only yng pos 182 42.973 21.507 1.594 45.833
value name-only,name+photo old neg 174 37.559 22.583 1.712 37.333
value name-only,name+photo old pos 174 44.661 22.997 1.743 44.333
value name-only,name+photo yng neg 164 42.240 20.371 1.591 42.667
value name-only,name+photo yng pos 164 41.026 19.506 1.523 41.000
value name+age,name-only old neg 216 38.110 20.683 1.407 38.167
value name+age,name-only old pos 216 43.174 19.819 1.349 45.333
value name+age,name-only yng neg 180 40.480 21.760 1.622 38.667
value name+age,name-only yng pos 180 43.567 21.225 1.582 44.667
value name+age,name+photo old neg 166 39.201 21.071 1.635 39.333
value name+age,name+photo old pos 166 41.946 21.045 1.633 45.000
value name+age,name+photo yng neg 198 36.768 21.424 1.523 36.667
value name+age,name+photo yng pos 198 45.813 20.633 1.466 48.833
value name+age+photo,name-only old neg 198 39.778 21.215 1.508 41.333
value name+age+photo,name-only old pos 198 41.402 21.501 1.528 43.667
value name+age+photo,name-only yng neg 194 41.136 21.291 1.529 41.500
value name+age+photo,name-only yng pos 194 42.711 20.591 1.478 45.333

This is probably not very meaningful because the means are beyond the evaluated individual (i.e., whether he’s old or young).

Finally, a tiny group by att.vlnc effect. This could reflect a pro-young bias, beyond the behavioral information provided about the targets (the “positive” factor)

mysumBy(value ~ group + att.vlnc, dt = mmm)

Yes, a tiny bias in favor of the young target. With eval.gb (i.e., the difference score), the effect had exactly the same (tiny) size. ok.

Moderation by language

First, a reminder about the language variable: Language. We asked: Which of the following is the most accurate? answers: [‘English is my native language’, ‘English is not my native langauge, but it is my primary language’, ‘English is not my native language, and not my primary language, but I am fluent in English’, ‘I understand some English’ ] coded as 1, 2, 3, and 4 (-999 selected a “decline to answer” button at the bottom of the page)

my.freq(allok$lang)

Let’s see whether the results are mighty different if we include only participants who reported that English was their native language.

We’ll jump to the ANOVA, to see whether anything of interest is detected here

aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$lang == 1), ], between = c("iat.order1", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))

The effect of “positive” (the group of the positively-portrayed target) was half of the small effect found with the whole sample. So, let’s see whether the effect becomes stronger the less people understand English:

mysumBy(IAT.gb ~ lang + positive, dt = allok)

Well, it seems that the effect was always in the expected direction, but clearly more tiny in the largest group, those who reported that English is their native language. Oddly, they also showed a relatively small induction effect (preference for the positively portrayed target individual). So, it is not entirely clear whether this moderation by language may reflect a stronger reliance on people’s groups than on their behaviors the less they know English.

The same for the self-report evaluation measure:

mysumBy(eval.gb ~ lang + positive, dt = allok)

The IAT pattern does not replicate at all. The largest effects are found in the two conditions that showed the smallest effect with the IAT.

ANOVA with language as a factor. For the IAT:

aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$lang %in% c(1, 2, 3, 4)), ], between = c("condition", "positive", "lang"), return = "nice", anova_table = list(es = "pes"))

Nothing. We lost the small bias effect (of the factor “positive”).

For self-report:

aov_ez(id = "session_id", dv = "eval.gb", data = allok[which(allok$lang %in% c(1, 2, 3, 4)), ], between = c("condition", "positive", "lang"), return = "nice", anova_table = list(es = "pes"))

Again, no effect at all, after losing the tiny effect of “positive”.

Let’s focus on the effect that showed the stronger pro-young bias with the IAT (name+age,name+photo):

mysumBy(IAT.gb ~ lang + positive, dt = allok[which(allok$condition == "name+age,name+photo"), ])

The Ns are very small, but it seems that the effect was found in all the language groups, other than lang=2 (probably random, due to small number of participants).

Moderation by age

First, here are the percentiles of the participants’ report age:

plot.percentile(allok$age)

Probably people under 50 (80%) would feel closer to the young person than to the old people.

We will start with the relation between age and preference for the young target over the old target, and a bunch of memory scores [macc is the accuracy rate by the person who was actually attributed with the behavior and the valence of the behavior, m.acc.byresp is the memory accuracy rate by the participant’s responseand the valence of the behavior, m.n.byresp is the number of responses of each type of response and the valence of the behavior]:

my.htmlTable(cornp(allok[, c("age", "eval.diff", "IAT.prf", "macc.o.neg", "macc.o.pos", "macc.y.neg", "macc.y.pos", "m.n.byresp.old.neg", "m.n.byresp.old.pos", "m.n.byresp.yng.neg", "m.n.byresp.yng.pos", "m.acc.byresp.old.neg", "m.acc.byresp.old.pos", "m.acc.byresp.yng.neg", "m.acc.byresp.yng.pos")]))
varName____ age____ eval.diff____ IAT.prf____ macc.o.neg____ macc.o.pos____ macc.y.neg____ macc.y.pos____ m.n.byresp.old.neg____ m.n.byresp.old.pos____ m.n.byresp.yng.neg____ m.n.byresp.yng.pos____ m.acc.byresp.old.neg____ m.acc.byresp.old.pos____ m.acc.byresp.yng.neg____
1 eval.diff 0.021
0.523
925
2 IAT.prf 0.007
0.831
925
0.237
< .001
925
3 macc.o.neg -0.015
0.65
905
0.04
0.233
905
-0.051
0.129
905
4 macc.o.pos -0.021
0.536
905
-0.067
0.043
905
-0.056
0.093
905
0.366
< .001
905
5 macc.y.neg -0.072
0.029
905
-0.059
0.076
905
-0.024
0.465
905
0.437
< .001
905
0.515
< .001
905
6 macc.y.pos 0.018
0.584
905
0.049
0.14
905
-0.066
0.049
905
0.492
< .001
905
0.327
< .001
905
0.332
< .001
905
7 m.n.byresp.old.neg 0.06
0.072
896
0.219
< .001
896
-0.007
0.835
896
0.443
< .001
896
-0.11
< .001
896
-0.341
< .001
896
0.202
< .001
896
8 m.n.byresp.old.pos 0.006
0.863
892
-0.167
< .001
892
-0.015
0.658
892
-0.176
< .001
892
0.465
< .001
892
0.082
0.014
892
-0.357
< .001
892
-0.129
< .001
885
9 m.n.byresp.yng.neg -0.016
0.63
898
-0.166
< .001
898
-0.005
0.891
898
-0.463
< .001
898
0.081
0.015
898
0.279
< .001
898
-0.197
< .001
898
-0.462
< .001
890
0.479
< .001
888
10 m.n.byresp.yng.pos 0.079
0.018
898
0.215
< .001
898
0.016
0.635
898
0.033
0.317
898
-0.414
< .001
898
-0.222
< .001
898
0.408
< .001
898
0.461
< .001
893
-0.392
< .001
887
-0.021
0.531
893
11 m.acc.byresp.old.neg -0.061
0.069
896
-0.122
< .001
896
-0.041
0.223
896
0.607
< .001
896
0.425
< .001
896
0.7
< .001
896
0.304
< .001
896
-0.368
< .001
896
-0.073
0.03
885
-0.092
0.006
890
-0.339
< .001
893
12 m.acc.byresp.old.pos -0.022
0.514
892
0.075
0.025
892
-0.044
0.192
892
0.501
< .001
892
0.539
< .001
892
0.469
< .001
892
0.657
< .001
892
-0.015
0.663
885
-0.412
< .001
892
-0.304
< .001
888
-0.034
0.309
887
0.509
< .001
885
13 m.acc.byresp.yng.neg -0.045
0.178
898
0.074
0.027
898
-0.002
0.95
898
0.729
< .001
898
0.365
< .001
898
0.623
< .001
898
0.42
< .001
898
0.064
0.057
890
-0.278
< .001
888
-0.509
< .001
898
-0.153
< .001
893
0.649
< .001
890
0.618
< .001
888
14 m.acc.byresp.yng.pos -0.05
0.132
898
-0.135
< .001
898
-0.076
0.023
898
0.415
< .001
898
0.643
< .001
898
0.477
< .001
898
0.534
< .001
898
-0.188
< .001
893
0.015
0.651
887
-0.162
< .001
893
-0.482
< .001
898
0.578
< .001
893
0.596
< .001
887
0.518
< .001
893

Age was unrelated to any score. IAT and self-report were weakly related (as often found with age attitudes) The relations of the self-reported evaluation with a couple of the memory scores is probably random, considering that it was unrelated to other memory scores.

Let’s see whether restricting the sample to age<50 would change the results. The IAT anova:

aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$age < 50), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

The effect is similar, but it is so tiny that it is non-significant.

Only age>49:

aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$age > 49), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

The effect is large, but still non-significant (smaller sample). So, if anything, there is a larger effect with older participants, but it may also be due to random assignment to different conditions.

Age by condition:

mysumBy(age ~ condition, dt = allok, probs = c(0.25, 0.75))

It’s hard to tell.

Age as a covariate (IAT.gb ~ condition * positive + age):

ccc <- allok[, c("session_id", "IAT.gb", "condition", "positive", "age")]
ccc <- ccc[complete.cases(ccc), ]

library(car)
# Fit the linear model
lm_model <- lm(IAT.gb ~ condition * positive + age, data = ccc)

# Perform ANOVA
anova_result <- Anova(lm_model, type = 3)

# Print the ANOVA table
print(anova_result)
Anova Table (Type III tests)

Response: IAT.gb
                    Sum Sq  Df F value  Pr(>F)   
(Intercept)          0.646   1  4.3153 0.03805 * 
condition            2.459   4  4.1052 0.00265 **
positive             0.026   1  0.1768 0.67424   
age                  0.016   1  0.1072 0.74338   
condition:positive   0.983   4  1.6408 0.16186   
Residuals          136.868 914                   
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

But, I am not sure that we are expect age to really predict the preference between the positive and negative targets (by their behaviors), and therefore, just adding age as a covariate may not make much sense.

Moeration by paticipant’s Gender

First, recoded gender to male=0, female-1, and correlated with the evaluation measures and the memory scores

allok$gender.i <- ifelse(allok$gender == "f", 1, ifelse(allok$gender == "m", 0, NA))
my.htmlTable(cornp(allok[, c("gender.i", "eval.diff", "IAT.prf", "macc.o.neg", "macc.o.pos", "macc.y.neg", "macc.y.pos", "m.n.byresp.old.neg", "m.n.byresp.old.pos", "m.n.byresp.yng.neg", "m.n.byresp.yng.pos", "m.acc.byresp.old.neg", "m.acc.byresp.old.pos", "m.acc.byresp.yng.neg", "m.acc.byresp.yng.pos")]))
varName____ gender.i____ eval.diff____ IAT.prf____ macc.o.neg____ macc.o.pos____ macc.y.neg____ macc.y.pos____ m.n.byresp.old.neg____ m.n.byresp.old.pos____ m.n.byresp.yng.neg____ m.n.byresp.yng.pos____ m.acc.byresp.old.neg____ m.acc.byresp.old.pos____ m.acc.byresp.yng.neg____
1 eval.diff -0.066
0.049
887
2 IAT.prf -0.048
0.149
887
0.237
< .001
925
3 macc.o.neg 0.015
0.651
868
0.04
0.233
905
-0.051
0.129
905
4 macc.o.pos 0.091
0.008
868
-0.067
0.043
905
-0.056
0.093
905
0.366
< .001
905
5 macc.y.neg 0.069
0.043
868
-0.059
0.076
905
-0.024
0.465
905
0.437
< .001
905
0.515
< .001
905
6 macc.y.pos 0.011
0.751
868
0.049
0.14
905
-0.066
0.049
905
0.492
< .001
905
0.327
< .001
905
0.332
< .001
905
7 m.n.byresp.old.neg -0.039
0.252
859
0.219
< .001
896
-0.007
0.835
896
0.443
< .001
896
-0.11
< .001
896
-0.341
< .001
896
0.202
< .001
896
8 m.n.byresp.old.pos 0.099
0.004
856
-0.167
< .001
892
-0.015
0.658
892
-0.176
< .001
892
0.465
< .001
892
0.082
0.014
892
-0.357
< .001
892
-0.129
< .001
885
9 m.n.byresp.yng.neg 0.066
0.053
861
-0.166
< .001
898
-0.005
0.891
898
-0.463
< .001
898
0.081
0.015
898
0.279
< .001
898
-0.197
< .001
898
-0.462
< .001
890
0.479
< .001
888
10 m.n.byresp.yng.pos -0.035
0.299
861
0.215
< .001
898
0.016
0.635
898
0.033
0.317
898
-0.414
< .001
898
-0.222
< .001
898
0.408
< .001
898
0.461
< .001
893
-0.392
< .001
887
-0.021
0.531
893
11 m.acc.byresp.old.neg 0.043
0.211
859
-0.122
< .001
896
-0.041
0.223
896
0.607
< .001
896
0.425
< .001
896
0.7
< .001
896
0.304
< .001
896
-0.368
< .001
896
-0.073
0.03
885
-0.092
0.006
890
-0.339
< .001
893
12 m.acc.byresp.old.pos 0.009
0.791
856
0.075
0.025
892
-0.044
0.192
892
0.501
< .001
892
0.539
< .001
892
0.469
< .001
892
0.657
< .001
892
-0.015
0.663
885
-0.412
< .001
892
-0.304
< .001
888
-0.034
0.309
887
0.509
< .001
885
13 m.acc.byresp.yng.neg -0.023
0.504
861
0.074
0.027
898
-0.002
0.95
898
0.729
< .001
898
0.365
< .001
898
0.623
< .001
898
0.42
< .001
898
0.064
0.057
890
-0.278
< .001
888
-0.509
< .001
898
-0.153
< .001
893
0.649
< .001
890
0.618
< .001
888
14 m.acc.byresp.yng.pos 0.02
0.554
861
-0.135
< .001
898
-0.076
0.023
898
0.415
< .001
898
0.643
< .001
898
0.477
< .001
898
0.534
< .001
898
-0.188
< .001
893
0.015
0.651
887
-0.162
< .001
893
-0.482
< .001
898
0.578
< .001
893
0.596
< .001
887
0.518
< .001
893

Gender is unrelated to anything.

IAT

The main IAT anova, among women only:

aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$gender == "f"), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

A small effect of the condition. It is probably not very interesting, but let’s see it:

mysumBy(IAT.gb ~ condition, dt = allok[which(allok$gender == "f"), ])

Very small effects of the induction, and the differences are hard to interpret.

The same ANOVA with males:

aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$gender == "m"), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

The men showed the bias, with a stronger effect and a smaller sample. Perhaps because they matched on gender (i.e., the targets were males). We haven’t tested that in the original study.

Means of IAT preference for the positively-portrayed individual by condition, males only:

mysumBy(IAT.gb ~ condition + positive, dt = allok[which(allok$gender == "m"), ])

The bias shows in the baseline condition, and in the two conditions that showed photos in the measurement.

Adding gender to the main ANOVA, as a factor.

aov_ez(id = "session_id", dv = "IAT.gb", data = allok[which(allok$gender %in% c("m", "f")), ], between = c("gender", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))

Gender does not moderate anything, but it does make the bias effect significant.

Self-report

The main anova, among women only:

aov_ez(id = "session_id", dv = "eval.gb", data = allok[which(allok$gender == "f"), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

Nada.

Men only:

aov_ez(id = "session_id", dv = "eval.gb", data = allok[which(allok$gender == "m"), ], between = c("condition", "positive"), return = "nice", anova_table = list(es = "pes"))

The bias effect is as strong as with the IAT.

Let’s see it by condition:

mysumBy(eval.gb ~ condition + positive, dt = allok[which(allok$gender == "m"), ])

The bias shows in all the conditions, other than [name-only,name+photo], and it is the strongest in [name+age+photo,name-only].

Adding gender to the main ANOVA, as a factor.

aov_ez(id = "session_id", dv = "eval.gb", data = allok[which(allok$gender %in% c("m", "f")), ], between = c("gender", "condition", "positive"), return = "nice", anova_table = list(es = "pes"))

Gender does not moderate anything, and the bias effect remains significant, but tiny.

Sanity checks

Compare conditions

We saved the conditions of the study within different tasks. Other times, it is possible to infer the condition from the data. For example, if we saved a photo data in the IAT, then we can infer that the paricipant observed a photo. There could be some inconsistencies due to technical problems. I think that the main reason is refreshing the page. It is supposed to cause an error an kill the study, but perhaps clicking Back in the browser, sometimes restarts the study session, possibly with different conditions. In this section, we will examine whether the conditions saved in the sample are consistent.

Induction.

In the induction task, we saved the condition by which we determined what to show to participants. Let’s make sure it was the same condition as the one recorded at the begining of the task.

table(indPart1Cond = allok$indPart1Cond, part1Cond = allok$part1Cond, exclude = NULL)
                part1Cond
indPart1Cond     name-only name+age name+age+photo
  name-only            349        0              0
  name+age               0      380              0
  name+age+photo         0        0            196

Perfect.

From the induction, we can infer whether the old or the yng target individual was the positive or the negative target, based on the number of positive (and negative) behaviors presented in the induction task

table(ind.oGood = allok$ind.oGood, oldVlnc = allok$oldVlnc, exclude = NULL)
         oldVlnc
ind.oGood   b   g
    FALSE 459   0
    TRUE    0 466

Good. and for the young:

table(ind.yGood = allok$ind.yGood, oldVlnc = allok$oldVlnc, exclude = NULL)
         oldVlnc
ind.yGood   b   g
    FALSE   0 466
    TRUE  459   0

Good.

I also counted the number of positive and negative behaviors presented in the induction, based on the data recorded in that task. Old target:

table(old.bvs = allok$o.bvs, young.bvs = allok$y.bvs, exclude = NULL)
       young.bvs
old.bvs 4-8 8-4
    4-8   0 459
    8-4 466   0

All good.

Matching

In the matching task, we saved the names and face identity of the targets, based on what has been shown. It was also saved at the beginning of the task, after assigning those stimuli. Let’s test for consistency:

table(match.oName = allok$match.oName.y, oName = allok$oName, exclude = NULL)
           oName
match.oName Andy Chris Daniel Jack James John Matt Michael Nick
    Andy     118     0      0    0     0    0    0       0    0
    Chris      0   105      0    0     0    0    0       0    0
    Daniel     0     0    105    0     0    0    0       0    0
    Jack       0     0      0   96     0    0    0       0    0
    James      0     0      0    0    97    0    0       0    0
    John       0     0      0    0     0   88    0       0    0
    Matt       0     0      0    0     0    0  125       0    0
    Michael    0     0      0    0     0    0    0      92    0
    Nick       0     0      0    0     0    0    0       0   99

All seems fine.

table(match.yName = allok$match.yName.y, oName = allok$yName, exclude = NULL)
           oName
match.yName Andy Chris Daniel Jack James John Matt Michael Nick
    Andy     105     0      0    0     0    0    0       0    0
    Chris      0   101      0    0     0    0    0       0    0
    Daniel     0     0    104    0     0    0    0       0    0
    Jack       0     0      0  112     0    0    0       0    0
    James      0     0      0    0   102    0    0       0    0
    John       0     0      0    0     0   99    0       0    0
    Matt       0     0      0    0     0    0  103       0    0
    Michael    0     0      0    0     0    0    0      95    0
    Nick       0     0      0    0     0    0    0       0  104

All seems fine.

IAT

From the IAT, we can learn whether photos or names were presented.

table(IAT.stims = allok$iat.vCond, stimCond = allok$part3Cond, exclude = NULL)
         stimCond
IAT.stims name-only name+photo
    photo         0        351
    word        574          0

All ok.

Attribute ratings

From the attribute questionnaire, we saved the Photo identity chosen (also saved in the matching task)

table(att.old.id = allok$att.man.id.o, match.old.id = allok$match.oFace.y, exclude = NULL)
          match.old.id
att.old.id old_a old_b old_c old_d old_e old_f
     old_a   140     0     0     0     0     0
     old_b     0   162     0     0     0     0
     old_c     0     0   133     0     0     0
     old_d     0     0     0   154     0     0
     old_e     0     0     0     0   170     0
     old_f     0     0     0     0     0   166
table(att.yng.id = allok$att.man.id.y, match.yng.id = allok$match.yFace.y, exclude = NULL)
          match.yng.id
att.yng.id young_a young_b young_c young_d young_e young_f
   young_a     162       0       0       0       0       0
   young_b       0     151       0       0       0       0
   young_c       0       0     173       0       0       0
   young_d       0       0       0     150       0       0
   young_e       0       0       0       0     152       0
   young_f       0       0       0       0       0     137

And the names:

table(att.old.name = allok$att.man.name.o, old.name = allok$oName, exclude = TRUE)
            old.name
att.old.name Andy Chris Daniel Jack James John Matt Michael Nick
     Andy     118     0      0    0     0    0    0       0    0
     Chris      0   105      0    0     0    0    0       0    0
     Daniel     0     0    105    0     0    0    0       0    0
     Jack       0     0      0   96     0    0    0       0    0
     James      0     0      0    0    97    0    0       0    0
     John       0     0      0    0     0   88    0       0    0
     Matt       0     0      0    0     0    0  125       0    0
     Michael    0     0      0    0     0    0    0      92    0
     Nick       0     0      0    0     0    0    0       0   99
table(att.yng.name = allok$att.man.name.y, yng.name = allok$yName, exclude = TRUE)
            yng.name
att.yng.name Andy Chris Daniel Jack James John Matt Michael Nick
     Andy     105     0      0    0     0    0    0       0    0
     Chris      0   101      0    0     0    0    0       0    0
     Daniel     0     0    104    0     0    0    0       0    0
     Jack       0     0      0  112     0    0    0       0    0
     James      0     0      0    0   102    0    0       0    0
     John       0     0      0    0     0   99    0       0    0
     Matt       0     0      0    0     0    0  103       0    0
     Michael    0     0      0    0     0    0    0      95    0
     Nick       0     0      0    0     0    0    0       0  104

Let’s also make sure all the name combinations occurred, but never using the same name for both targets:

table(old.name = allok$oName, yng.name = allok$yName, exclude = TRUE)
         yng.name
old.name  Andy Chris Daniel Jack James John Matt Michael Nick
  Andy       0    18     18   11    15   13   13      16   14
  Chris     14     0     12   14    14   13   12      13   13
  Daniel    12    10      0   18    12   10   17       8   18
  Jack      14     8      4    0    15   16   15      11   13
  James     13    15     15   11     0   13   13       9    8
  John       8     7     17    8     8    0    9      13   18
  Matt      14    21     18   18    14   17    0      11   12
  Michael   13    14      9   14    13    8   13       0    8
  Nick      17     8     11   18    11    9   11      14    0

ok.

In the attributes questionnaire, we also saved the evaluation condition (with or without photos)

table(part3Cond = allok$part3Cond, attCond = allok$attPart3Cond, exclude = NULL)
            attCond
part3Cond    name-only name+photo
  name-only        574          0
  name+photo         0        351

ok ok.

Recoding

One odd thing in the correlation matrix above was that the correlation between the IAT and self-reported preference changed when we correlated preference for the positively-portrayed target vs. when we correlated the preference for the young target. one was a recoding of the other (dependended on the induction condition), so it seems reasonable to expect the same correlation. To verify that no mistake was made, let’s see those correlations within each induction condition.

First, when the old target was the positively-portrayed target.

my.htmlTable(cornp(allok[which(allok$oldVlnc == "g"), c("IAT.gb", "IAT.prf", "eval.diff", "eval.gb")]))
varName____ IAT.gb____ IAT.prf____ eval.diff____
1 IAT.prf -1
< .001
466
2 eval.diff -0.213
< .001
466
0.213
< .001
466
3 eval.gb 0.213
< .001
466
-0.213
< .001
466
-1
< .001
466

Second, when the old target was the negatively-portrayed target:

my.htmlTable(cornp(allok[which(allok$oldVlnc == "b"), c("IAT.gb", "IAT.prf", "eval.diff", "eval.gb")]))
varName____ IAT.gb____ IAT.prf____ eval.diff____
1 IAT.prf 1
< .001
459
2 eval.diff 0.113
0.016
459
0.113
0.016
459
3 eval.gb 0.113
0.016
459
0.113
0.016
459
1
< .001
459

The important correlations were within the IAT scores and within the eval (self-report) scores. The -1 and 1 correlations show that the recoding was as expected. I am not entirely sure wha the reason for the difference in correlations is, when looking at the whole sample:

my.htmlTable(cornp(allok[, c("IAT.gb", "IAT.prf", "eval.diff", "eval.gb")]))
varName____ IAT.gb____ IAT.prf____ eval.diff____
1 IAT.prf -0.021
0.52
925
2 eval.diff -0.008
0.816
925
0.237
< .001
925
3 eval.gb 0.171
< .001
925
-0.039
0.237
925
0.001
0.974
925

.237 vs. 0.171. Perhaps this is related to SD. Let’s verify that it doesn’t happen with Spearman correlation (i.e., the ranking stays the same)

my.htmlTable(cornp(allok[, c("IAT.gb", "IAT.prf", "eval.diff", "eval.gb")], type = "spearman"))
varName____ IAT.gb____ IAT.prf____ eval.diff____
1 IAT.prf 0.002
0.962
925
2 eval.diff 0.025
0.45
925
0.233
< .001
925
3 eval.gb 0.164
< .001
925
-0.027
0.411
925
0.014
0.66
925

No. So it’s unrelated to SD.

Let’s simulate data and implement the same recoding, to make sure this can happen

df <- data.frame(sid = c(1:1000), one = rnorm(n = 1000, mean = 0.33, sd = 0.397), cond = rep(c(2, 1), 500))
df$two <- rnorm(n = nrow(df), mean = 3.779, sd = 53.916) + (df$one * 25)

df$one2 = ifelse(df$cond == 1, df$one, -1 * df$one)
df$two2 = ifelse(df$cond == 1, df$two, -1 * df$two)

my.htmlTable(cornp(df))
varName____ sid____ one____ cond____ two____ one2____
1 one -0.006
0.845
1000
2 cond -0.002
0.956
1000
-0.029
0.356
1000
3 two -0.039
0.214
1000
0.179
< .001
1000
0.026
0.414
1000
4 one2 0.021
0.513
1000
0.031
0.334
1000
-0.634
< .001
1000
-0.029
0.364
1000
5 two2 -0.042
0.181
1000
-0.011
0.739
1000
-0.177
< .001
1000
0.015
0.638
1000
0.249
< .001
1000

The comparison of interest is between the correlation of one with two and the correlation of one2 with two2. We can see that these are not the same correlations.

Summary

  • The age bias in the IAT was quite small, in comparison to previous studies.

    • One possible reason is the addition of names. We used only the photos in previous research.
    • Another possible reason is that in the present research we never presented photos in both encoding and measurement, unlike previous research,
    • Another possible reason is that the 8-4 vs. 4-8 induction condition was not ideal for obtaining strong results.
      • That seems relatively unlikely because in some follow-up studies (mostly with gender, but also with age), there was no problem in finding strong bias effects, despite using various different behavioral information rates.
  • in none of the 5 conditions, the age-bias was strong in both IAT block-order conditions. This is further evidence that we did not obtain strong and reliable age bias with the IAT.

    • The closest to a consistent age-bias effect in the IAT was in the [name+age, name+photo] condition: a clear and strong age-bias when the incompatible pairing (bad-target+good-words) was the first pairing in the IAT, and a small age-bias when the compatible pairing was first. In fact, the [name+age, name+photo] condition was the only that showed a significant age-bias effect (d = 0.40) in the statistical analysis, with two other conditions (name-only,name+photo, name+age,name-only) showing a non-significant age bias of d=0.27.
  • There was some surprising age bias in the self-reported evaluation: in the [name-only,name-only] (d=0.33) and [name+age+photo,name-only] (d=0.49) conditions. Oddly, these conditions did not show age-bias in the IAT. It is difficult to understand what happened here, but might be worth replicating.

  • Men showed a stronger pro-young bias than women, but gender was not a significant moderator.

  • People had the best memory for behaviors attributed to the young target, followed by the old target, and then novel behaviors.

    • That difference did not occur in the [name-only, name-only] baseline condition. People had more accurate memory, the more individuating information appeared in the induction task. We did not find evidence that individuating information in the memory test (retrieval) helped, but we also did not have conditions that provided those cues in both encoding and retrieval (not sure there is a reason to expect that cues at retrieval that did not appear in encoding would improve memory).
  • Importantly, however, people had a response-bias: they were more likely to attribute behaviors to the “young” person. So, rather than better memory for the young target, the effect was a bias to respond “young”. More sophisticated analysis is needed to understand effects on memory quality (sensitivity).

    • For negative behaviors, the “none” response was less likely and the “young” response was more likely when photos appeared in the memory test. For positive behaviors, the “none” response was less likely and the “old” response was more likely when photos appeared in the memory test. I am not sure what that might mean.

Conclusions

  • The age-bias was much smaller than in previous studies. That might be a finding, if we can detect the reason.
    • Perhaps we should try repeating this study with a condition that is similar to previous research: only photos in induction and evaluation, and no matching task.
  • We should probably test whether the age-bias effects in self-reported evaluation were a statistical fluke. For that, it might be good to repeat the condition that showed the strongest such bias [name+age+photo,name-only].